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This  report  is  part  of  the  renewal  proposal  submitted  to  AFOSR.  It  summarizes  the  results 
we  have  obtained  so  far. 

Attentional  manipulations 

Attention  is  the  first  cognitive  faculty  we  have  explored  in  our  attempt  to  understand 
covert  interventions.  On  the  one  hand,  there  is  no  better  way  to  implement  covertness 
than  designing  interventions  that  are  invisible  even  to  the  adversary  operator’s  attentional 
system.  In  this  case,  the  interventions  are  completely  hidden  and  therefore  can  potentially 
cause  most  and  long-term  damage.  On  the  other  hand,  a  large  body  of  evidence  in  the 
field  of  psychology  has  shown  that  attention  is  a  fragile  function  that  is  subject  to 
exploitation  and  manipulation.  Our  exploration  with  attention-based  covertness  started 
with  a  taxonomy  of  attention.  A  recent  theoretical  breakthrough  of  attention  research  is 
the  notion  that  there  exist  different  types  of  attention,  each  of  which  is  subserved  by 
different  brain  regions  and  is  sensitive  to  different  variables  (Fan,  McCandliss,  Sommer, 
Raz,  &  Posner,  2002;  Posner,  2004).  Alerting  (for  general  preparatory  attention), 
orienting  (for  selecting  subset  of  information  for  further  processing),  and  executive 
control  (for  monitoring  and  resolving  conflict  in  planning  and  decision-making)  have 
been  distinguished  at  both  functional  and  brain  levels.  Equipped  with  the  taxonomy,  we 
suggested  that  each  type  of  attention  could  be  subject  to  different  exploitations  for  the 
purpose  of  covertness.  We  systematically  explored  and  identified  a  set  of  manipulations 
that  could  be  used,  including  attentional  blink,  inhibition  of  return,  change  blindness,  the 
order  effect,  negative  priming,  and  etc.  We  conducted  studies  to  examine  the  effect  of 
parameter  changes  on  inducing  covertness  in  order  to  find  the  optimal  delivery  schemes 
(Fan  et  al.,  2009;  Sun,  Wang,  Zhang,  &  Smith,  2008;  Wang  &  Fan,  2007). 

Stealth  and  disruption  with  IMPs 

As  a  preliminary  step  toward  abduction-based  covertness,  we  conducted  a  study  to 
examine  how  a  human  operator  digests  unexpected  interventions  and  adjusts  his  level  of 
suspicion.  The  study  utilized  Interface  Manipulation  Protocol  (IMP),  which  was 
developed  by  our  collaborators  at  AFRL  at  Rome.  The  toolbox  of  IMP  contained  dozens 
of  possible  intervention  types  that  could  be  delivered  to  the  adversary  computers  to  cause 
disruption  with,  for  example,  keyboard  and  mouse  operations.  How  to  configure  the 
chain  of  IMPs  (e.g.,  when  to  deliver  what  IMP  for  how  long?)  to  cause  maximal 
disruption  with  minimal  suspicion  raised  a  challenge. 

In  the  study,  subjects  were  instructed  to  type  in  sequences  of  random  numbers  as 
prompted  (Figure  1),  where  3  types  of  IMPs  were  silently  delivered  (by  hijacking  the 
subject’s  keyboard).  Subjects  were  then  asked  to  evaluate  the  “reliability”  of  the  input 
device. 


Target 

Sequence: 

Responses: 


Figure  1.  Subjects  were  required  to  reproduce  the  target  sequence.  Errors  were 
prompted  in  red  color  and  need  to  be  corrected  with  extra  keystrokes.  Errors 
could  included  “IMP  errors”  (produced  deliberately  by  IMPs)  and  “genuine 
errors”  (subjects’  own  typos). 


The  study  included  several  independent  variables,  including: 

•  3  types  of  IMPs 

o  non-responsive  key:  when  a  key  is  typed  nothing  shows  up,  so  the  subject 
has  to  retype  to  correct; 

o  repetitive  key:  when  a  key  is  typed  the  key  shows  up  twice  (e.g.,  typing 
“3”  and  “33”  shows  up),  so  the  subject  has  to  erase  the  extra  number; 
o  altered  key:  when  a  key  is  typed  a  different  key  shows  up  (e.g.,  typing  “3” 
and  “4”  shows  up),  so  the  subject  has  to  erase  the  wrong  key  and  retype. 

•  4  types  of  delivery  themes 

o  Pure:  only  one  type  of  IMPs  is  delivered  in  a  particular  trial, 
o  Mixed:  multiple  types  of  IMPs  are  delivered  in  a  particular  trial, 
o  Clumped:  IMPs  are  delivered  consecutively, 
o  Dispersed:  IMPs  are  delivered  sparsely. 

•  4  levels  of  IMP  delivery  rates  (10%,  20%,  30%,  40%  of  the  garget  characters  are 
affected  by  IMPs). 


There  were  two  major  dependent  measures.  Stealth  (covertness)  was  measured  by  the 
subjective  evaluations  of  the  reliability  of  the  input  device.  Higher  evaluation  scores 
indicated  higher  tolerance  of  IMPs  and  therefore  less  suspicion.  Disruption  was  measured 
by  the  number  of  extra  keystrokes  (“ExtraKS”)  required  to  complete  the  sequence 
(excluding  the  extra  keystrokes  directly  caused  by  IMPs).  Higher  scores  of  ExtraKS 
indicated  more  severe  disruptions  to  the  performance.  The  goal  was  to  find  an  optimal 
delivery  strategy  that  caused  maximal  disruption  with  maximal  stealth. 


One  main  result  of  the  study  is  shown  in  Figure  2,  which  depicts  the  effect  of  delivery 
themes  on  stealth  and  disruption.  It  is  clear  that  in  terms  of  stealth  the  mixed-clumped 
delivery  (IMPs  with  mixed  types  are  delivered  continuously)  is  the  best  and  that  in  terms 
of  disruption  the  pure-dispersed  delivery  (IMPs  with  the  same  type  are  delivered 
sparsely)  is  the  best.  Further  analysis  shows  that  if  we  combine  the  two  dependent 
measures,  the  pure-dispersed  delivery  has  the  highest  effectiveness  score. 


Stealth 


Disruption 


Figure  2.  The  effect  of  delivery  themes  (Base:  no  IMP  was  delivered;  MC:  IMPs 
were  delivered  in  mixed-clumped  fashion;  MD:  mixed-dispersed;  PC:  pure- 
clumped;  PD:  pure-dispersed). 

DUR1P  2008  award 


We  are  also  happy  to  report  that  we  have  obtained  a  DURIP  2008  award  thanks  to  the 
support  from  AFOSR  associated  with  this  project.  The  award  allowed  us  to  purchase  an 
eye-tracking  system  (Smart-Eye,  www.smarteve.sel  that  could  be  integrated  with  our 
existing  neuroimaging  system  (128-channel  EGI  system,  www.egi.coml  and  powerful 
experimental  design  system  (E-prime,  www.pstnet.coml.  The  integrated  system  (Figure 
3),  the  first  in  the  US,  is  capable  of  collecting  temporally  synchronized  high-resolution 
eye-movement  data  and  neural  electroencephalography  (EEG)  data,  which  is  ideal  for  the 
purpose  of  the  current  project,  where  one  goal  is  to  monitor  human  suspicion  and  study 
how  suspicion  is  modulated  by  interventions. 
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Figure  3.  Left:  the  integrated  experiment  system  (a  head  wearing  a  128-channel 
EEG  net,  red  arrows  points  to  the  two  eye-tracking  cameras).  Right:  via  the  UT- 
Tracking  package  we  developed,  we  integrate  the  systems  together  and  gain 
unique  capabilities  including  real-time  fixation  analyses,  AOI  association, 
interaction  /  gaze  control. 


Recently,  we  have  developed  a  software  package  named  UT-tracking  to  fully  realize  the 
function  of  the  integrated  system  (Figure  3).  The  package  allows  more  seamless 


integration  of  different  sub-systems  and  offers  unprecedented  capabilities  for  multi-mode 
data  collection  and  analysis.  Some  capabilities  include: 

•  Real-time  eye  /  mouse  movement  tracking  and  fixation  analyses  (independent  or 
integrated  tracking). 

•  AOI  (area  of  interest)  association. 

•  Real-time  eye  gaze  interaction  and  gaze  control. 

•  Synchronization  with  higher-level  behavioral  measures. 

•  Easy  integrations  with  neuroimaging  data  collection  -  fixation  and  AOI  results 
can  be  used  to  segment  EEG. 


